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Abstract 

Background: The human resources for health crisis has highlighted the need for high-level public health education 
to add specific capacities to the workforce. Recently, it was questioned whether Master of Public Health (MPH) 
training prepared graduates with competencies relevant to low- and middle-income countries (LMICs). This study 
aims to examine the influence of the MPH programs geared towards LMICs offered in Vietnam, China, South Africa, 
Mexico, Sudan, and the Netherlands on graduates' careers, application of acquired competencies, performance at 
the workplace, and their professional contribution to society. 

Methods: A self-administered questionnaire was sent to graduates from six MPH programs. Frequency distributions 
of the answers were calculated, and a bivariate analysis and logistic regression of certain variables was performed. 

Results: The response rate was 37.5%. Graduates reported change in leadership (69%), in technical position (69%), 
acquiring new responsibilities (80%), and increased remuneration (63%); they asserted that MPH programs 
contributed significantly to this. Graduates' attribution of their application of 7 key competencies 'substantially to 
the MPH program' ranged from 33% to 48%. Of the 26 impact variables, graduates attributed the effect they had 
on their workplace substantially to the MPH program; the highest rated variable ranged from 31% to 73% and the 
lowest ranged from 9% to 43%. Of the 10 impact variables on society, graduates attributed the effect they had on 
society substantially to the MPH program; for the highest rated variable (13% to 71%); for the lowest rated variable 
(4% to 42%). Candidates' attribution of their application of acquired competencies as well as their impact at the 
workplace varied significantly according to institution of study and educational background. 

Conclusions: This study concludes that these MPH programs contribute to improving graduates' careers and to 
building leadership in public health. The MPH programs contribute to graduates' application of competencies. MPH 
programs contribute substantially towards impact variables on the workplace, such as development of research 
proposals and reporting on population health needs, and less substantially to their impact on society, such as 
contributing equitable access to quality services. Differences reported between MPH programs merit further study. 
The results can be used for curriculum reform. 
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Background 

The human resources for health crisis, i.e., the severe 
shortage of human resources in 57 low- and middle- 
income countries (LMICs), has highlighted the need for 
high-level public health education to add specific capacities 
to the workforce [1-5]. However, questions have been 
posed as to whether Master of Public Health (MPH) train- 
ing prepared graduates with competencies relevant for 
LMICs [6-8]. These questions have also been raised in 
high-income countries [9,10] and were probably influenced 
by the general debate on the impact of higher education 
[11-15]. In addition, WHO identified evaluation of the edu- 
cation of health professionals as a knowledge gap [16]. 

Measuring outcome and impact of educational programs 
is fraught with methodological difficulties [14,15,17]. 
Blomeke pointed to the dearth of literature measuring 
competencies of students and graduates in higher edu- 
cation, especially internationally comparable measure- 
ments [15]. A systematic review of Master's in Health 
and Health Care programs showed that outcome was 
usually measured through alumni surveys. In these alumni 
surveys, however, no questions on whether graduates 
attributed their advancement in their career or their appli- 
cation of competencies to the Master's program were in- 
cluded. Other methods used, though less often, were focus 
group discussions, employer surveys, and semi- or un- 
structured interviews [18]. Furthermore, although alumni 
were sometimes asked what they accomplished in their 
work, these questions were open-ended and did not 
address outcome or impact indicators. Impact on the 
workplace was measured in four studies [19-22] and im- 
pact on society was only reported in two studies [20,22]. 
Impact was not measured in a systematic manner in any 
of these studies. Self-reported competency and aca- 
demic outcome by students and graduates is a valid 
measure for higher education learning as shown by earl- 
ier studies [15,23-27]. 

Outcome in this study is defined as the application of 
competencies and as the effects on career, such as in- 
crease in leadership, new responsibilities, change in pos- 
ition, and increase in remuneration. Impact in this study 
is defined as impact on the workplace, e.g., "developed a 
study or a research proposal" and impact on sector or 
society, e.g., "contributed to equitable access to quality 
services". Competencies for the MPH and impact vari- 
ables on work and society were jointly constructed and 
validated prior to the study. The designed competencies 
were based on the competencies and learning objectives 
of the sbc participating institutions offering MPH pro- 
grams and the set of competencies of the Council on Link- 
ages Between Academia and Public Health Practice as a 
reference. The competencies and impact variables were 
validated with experts in the field and alumni in the six 
different countries [28]. 



The aim of this study was to analyze the influence of 
the MPH programs on graduates' careers and their lead- 
ership, on application of competencies acquired in the 
MPH program as well as on impact at the workplace 
and on their contribution to society. 



Methods 

This is a cross-sectional study of graduates from sbc 
MPH programs: Hanoi School of Public Health, Vietnam 
(HSPH), School of Public Health Fudan University, China 
(SPHFU), School of Public Health University of Western 
Cape, South Africa (SPHUWC), National Institute of Pub- 
lic Health Mexico, Mexico (INSP), University of Medical 
Sciences and Technology, Sudan (UMST), and the Royal 
Tropical Institute, Netherlands (KIT). All offer MPH 
programs geared towards LMICs. HSPH and KIT offer 
fulltime programs, SPHFU offers a part-time program 
and, since 2010, a fuHtime program. At SPHUWC, INSP 
and UMST students can follow the programs full- and 
part-time. 

An anonymous self-administered questionnaire was 
designed, based on the analytical framework of the sys- 
tematic review by Zwanikken and a previous question- 
naire [18,29]. As attribution was rarely addressed in 
articles reviewed [18], specific questions were asked re- 
garding the graduates' attribution of competencies and 
impact variables to the MPH program. The range of rat- 
ings was kept small to avoid the recognized tendency 
for respondents to repeat a rating where the range is 
wider (Additional file 1) [24,26]. 

The questionnaire was pretested with graduates from 
different years in all countries and revised, based on 
comments received. In Vietnam, Mexico, and China the 
questionnaire was translated into the national language 
and translated back to check for consistency of the 
translation. The questionnaire was administered by each 
institution through free online tools or through email 
(Table 1). Graduates were reminded two times by email, 
or by telephone. The questionnaire targeted graduates 

Table 1 Approaching graduates and tools used 

Institution How graduates Tool for filling in 

were approached questionnaire 

By email/reminder Questionnaire send through 
by telephone email 



HSPH (Vietnam) 

SPHFU (China) 

SPHUWC (South 
Africa) 

INSP (Mexico) 
UMST (Sudan) 



By telephone 
By email 

By email 



www.sojump.com 
www.surveymonkey.com 

Webserver of the institute 



By email/telephone Questionnaire send through 
email or hard copy 



KIT (The Netherlands) By email 



www.surveymonkey.com 
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from the MPH programs of the six participating institu- 
tions from 2005-2010, in total 1,187 graduates, to allow 
sufficient time for graduates to have applied their newly 
gained competencies. The questionnaire was online 
throughout November 2012 to February 2013. 

Prior to embarking on the study, the ethics committees 
of the six participating institutions, i.e., the University of 
Western Cape Senate Research and Ethics Committee, 
Hanoi School of Public Health Ethics Committee, Fudan 
University School of Public Health Institutional Review 
Board, Sudan Medical and Scientific Research Institute 
Ethical Clearance Committee, National Institute of Public 
Health Mexico Ethics Committee, and the Royal Tropical 
Institute Research Ethics Committee, granted ethical ap- 
proval for the study. 

Data analysis 

The answers from all institutions were analyzed using 
Microsoft Excel 2013 and SPSS 21. Descriptive and bi- 
variate analysis of specific variables was performed. 

Logistic regression was performed to examine whether 
a medical doctor background (yes/no), additional degree 
(yes/no), institution, gender, time of graduation (2005- 
2007 or 2008-2010), and age (in years) can predict change 
in leadership level (yes/no), change in technical position 
(yes/no), change to position involving more responsibility 
(yes/no), increase in remuneration (yes/no), and/or a 
change to another employer (yes/no). 

We performed analysis of variance (ANOVA) and ana- 
lysis of covariance (ANCOVA) to examine which of the 
aforementioned predictor variables yield a significant 
contribution to the perceived extent to which MPH con- 
tributed to a change in leadership, technical position, 
employer, and/or increase in remuneration. 

The questionnaire included a component on the extent 
to which MPH contributed to application of acquired com- 
petencies (Cronbach's a = 0.957), to graduates' performance 
at the work place (Cronbach's a = 0.954), and to their 
contribution to society (Cronbach's a = 0.940). For each of 
these three components, exploratory factor analysis using 
generalized least squares estimation was performed to 
compute factor scores following the Anderson-Rubin 
method [30]. These standardized factor scores, having a 
mean of zero and standard deviation of approximately 
one, were used as response variables in ANOVA and 
ANCOVA to examine which of the aforementioned pre- 
dictor variables yield a significant contribution to a 
higher extent of application of acquired competencies, 
better performance of graduates at the workplace, and 
an increased impact on society. For all ANOVAs and 
ANCOVAs, Eta-squared (i/^) was used as measure of ef- 
fect size. Values of 0.01 indicate small effects, values of 
0.06 indicate medium size effects, and values of 0.14 are 
indicative of large effects [30]. 



While Anderson-Rubin factor scores are generally some- 
what more precise than simple sum or average scores, a 
drawback of these factor scores is that one cannot use 
them to assess the degree of contribution of the MPH pro- 
gram to each - application of competencies or workplace 
performance or impact on society - because the mean of 
the aforementioned factor scores is zero. Therefore, for the 
latter, a proportion (i.e., a value somewhere between 0 and 
1) was calculated for each of these three components; we 
divided the number of "substantial attribution" responses 
by the number of items in the component. We then per- 
formed between-subjects-by-within-subjects ANOVA, 
treating the three components as within-subjects factors 
and treating institution as the between-subjects factor. 

Results 

Response rate and demographics of respondents 

The overall response rate was 37.5%. The highest re- 
sponse rate was obtained in Vietnam, and the lowest in 
Mexico and China, but overall the response rate between 
the institutions did not differ much (Table 2). 

Of the respondents, 50.8% were female and the me- 
dian year of birth was 1973, ranging from 1955-1997. 
Respondents had professional educational backgrounds 
in medicine (48.5%), Bachelor of Public Health (9.9%), 
nursing (7.4%), dentistry (6.3%), social science (3.8%), 
nursing/midwifery (2.3%), pharmacy (2%), BSc or BA 
(5.4%), or other (14.3%). Most respondents (69%) studied 
fuUtime. All graduates from HSPH and KIT studied full- 
time; all alumni from SPHFU studied part-time. Graduates 
had an average work experience of 9.2 years (median 
8 years) prior to the MPH, ranging from 0-30 years of 
work experience. 

Career and leadership 

The effect on career and leadership were measured by 
changes in level of employment, leadership, technical 
position, responsibilities, remuneration, and graduates' 
attribution to the MPH. 

Level of employment before MPH and currently 

Almost 50% of the alumni indicated that they worked in 
a cUnic (18.9%) or at district (13.9%) or state health pub- 
lic health service (14.1%) prior to the MPH. After gradu- 
ation, more than 50% of the graduates shifted towards 
working for the national Ministry of Health (12.7%), 
international non-governmental organizations (9.9%), a 
research institute (7%), or other (26%) (Figure 1). 

Only 5% [24] of the graduates reported working cur- 
rently outside their home country; this included two 
graduates who originated from a high income country. 
Fifteen of the graduates working outside their home coun- 
try worked within the region (i.e., Africa), while seven of 
them went to work in a high income country. No 
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Table 2 Response rate and demographics of respondents 





Number of 

1 CSIJUI lUCI 11.3 


Response 

1 aLC V ''^1 


Female 

1 lUCI IL3 \ /□/ 


Male 

1 lUCI ^ /Of 


Median 

yccii uyj 1 1 1 


Medical 


HSPH (Vietnam) 


153 


52 


57 


43 


1971 


64 


KIT (The Netherlands) 


86 


39 


43 


57 


1972 


47 


INSP (Mexico) 


61 


26 


71 


29 


1972 


56 


SPHFU (China) 


60 


26 


43 


57 


1975 


22 


SPHUWC (South Africa) 


50 


39 


54 


46 


1967 


22 


UMST (Sudan) 


35 


41 


48 


52 


1976 


54 


Total/Average 


445 


37.5 


51 


49 


1972 


49 



graduates from the schools of China and Vietnam worl<;ed 
outside their country, and only two from Mexico did so. 

Changes in leadership, technical position, responsibilities, 
and remuneration 

Graduates reported a change in leadership in the man- 
agement system after the MPH (mean: 69%, for ranges 
see Figure 2), a change in technical position or area of 
focus (mean: 69%; range: 57% to 85%), acquisition of 
new responsibilities (mean: 80%; range: 53% to 100%). 
More than half of the graduates (mean: 63%; range: 53% 
to 81%) reported an increase in remuneration, while 
32.7% remained the same. 

Graduates were asked to attribute the change in lead- 
ership, technical position, remuneration, and change of 
employer to the MPH graduation on a scale of 1-5 (in- 
significant to very significant). According to graduates, 
the MPH program contributed substantially to a change 
in leadership: nearly 76% responded this to be significant 



or very significant; as not enough people reported no 
change in leadership, a correlation could not be com- 
puted. The MPH program also contributed to change in 
employer: about 65% responded this to be significant or 
very significant. Furthermore, the MPH program was 
reported to contribute to change in technical position 
{p = 0.371, P <0.001) and to increase in remuneration 
{p = 0.430, P <0.001). 

Further training after the MPH 

More than half of the graduates (57%) reported having 
completed certified work-related training of 2 weeks or 
more, with a range from 1 to more than 10 courses. 
About a third (34%) took a further degree or diploma, 
other than short training courses, after the MPH. Gradu- 
ates reported to have taken a PhD (9%), diploma (9%), 
another Master degree (7%), a postgraduate diploma 
(1%), or other (10%). Graduates were eager to pursue 
further studies: 74% planned another degree. 



30% 
25X 
20X 
15% 
10% 
5% 
0% 



1-1- 



mm 




□ Work Before MPH 
■ CurrentWork 



4? <r 



J" 



Figure 1 Level of employment of graduates before MPH and currently (n = 445). 
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The extent to which the MPH program enabled the 
graduate to apply specific public health competencies in 
their work 

Seven core competencies were subdivided into detailed 
competencies. Graduates were asked to grade attribution 
to the MPH program per detailed competency (23 in 
total), scaled as follows: they did not use it/it was not 
part of their work; the MPH program did not enable the 
graduate; the MPH program enabled the graduate a little 
to apply; or enabled the graduate substantially to apply 
this competency in their work. Graduates stated that the 
MPH program enabled them substantially to apply the 
following core competencies: public health science skills 
including analytical assessment competencies (48%), lead- 
ership and systems thinking competencies (44%), context 
sensitive competencies (43%), and planning and manage- 
ment competencies (42%). About a third of the graduates 
stated that the MPH program enabled them substantially 
to apply the three following core competencies: communi- 
cation competencies (37%), community and inter-sectoral 
competencies (36%), and policy development competen- 
cies (33%). Strikingly, within all competencies, there were 
graduates that did not use the competency or it was not 
part of their work, with a range from 9 to 19%, of which 
the highest was the policy development competency (19%) 
(Table 3). 

The extent to which the MPH program enabled the 
graduate to impact on the workplace 

Graduates were asked to attribute their impact on the 
workplace to the MPH program as follows: they did not 
use it/it was not part of their work; the MPH program 
did not enable the graduate; the MPH program enabled 
the graduate a little; or enabled the graduate substan- 
tially to impact on workplace. Graduates stating that the 
MPH program enabled them substantially to impact on 



their workplace ranged from 60% to 20% for the 26 vari- 
ables. The three highest scored variables were "devel- 
oped a study or a research proposal" (mean 60%; range: 
73% to 32%), "reported and made recommendations on 
population health status or needs" (mean 43%; range: 
61% to 31%), and "made presentations at conferences" 
(mean 41%; range: 68% to 21%). The three lowest scored 
variables were "published or posted in popular (including 
electronic) media" (mean: 23%), "contributed to change in 
policy at one level higher than work institution" (mean: 
22%), and "contributed to writing a published chapter of a 
book" (mean: 20%) (Additional file 2). 

The extent to which the MPH program enabled the 
graduate to impact on society 

Graduates were asked to grade whether the MPH pro- 
gram enabled them to impact on society as follows: they 
did not use it/it was not part of their work; the MPH 
program did not enable the graduate; the MPH program 
enabled the graduate a little to impact; or enabled the 
graduate substantially to impact on society. Graduates 
stated that the MPH program enabled them substantially 
to impact on society with a range from 39% to 17% for 
the 10 variables. The three highest scored variables were 
"influenced better understanding of public health mea- 
sures amongst general population" (mean: 39%; range: 
71% to 13%), "contributed to equitable access to quality 
services" (mean 32%: range: 60% to 12%), and "contrib- 
uted to increased resource mobilization for disadvan- 
taged groups" (mean: 31%). The three lowest scored 
variables were "contributed to changes in policy or 
strategy in general" (mean: 25%), "contributed to equity/ 
pro-poor orientation towards health access at all levels" 
(mean: 25%), and "contributed to changed guidelines, 
regulations, ordinances beyond the workplace" (mean: 
17%) (Additional file 3). 
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Table 3 Enablement of application of specific public health competencies attributed to the MPH program as reported 
by graduates (n = 420)* 



Public health competencies/Attribution 
to MPH program* 


MPH enabled me 
substantially to apply 


MPH enabled me 
a little to apply 


Not due to MPH 


Not used/not 
part of my work 


Public health science skills including analytical 
assessment competencies 


48% 


34% 


9% 


9% 


Leadership and systems thinking competencies 


44% 


36% 


10% 


11% 


Context sensitive competencies 


43% 


33% 


12% 


12% 


Planning and management competencies 


42% 


33% 


9% 


15% 


Communication competencies 


37% 


39% 


13% 


11% 


Community and inter-sectoral competencies 


36% 


34% 


16% 


15% 


Policy development competencies 


33% 


36% 


12% 


19% 



•Missing: 25. 



Changes in position, competencies, and impact variables 
by predictor variable 

Graduates with a medical doctor background or other 
additional degree appear to be more lil<ely to change lead- 
ership after graduation (P <0.001). Candidates graduating 
from HSPH, SPHUWC, or UMST appear to be less likely 
to change leadership than graduates from other institutes 
(Additional file 4: Table S6). Graduates with an additional 
degree appear to be somewhat more likely to change tech- 
nical position. Graduates from HSPH, SPHFU, or UMST 
appear to be less likely to change technical position than 
graduates from other institutes (Additional file 4: Table 
S7). Graduates with an additional degree appear to be 
more likely to switch to a position involving more respon- 
sibility and graduates from INSP and KIT appear to be 
more likely to switch to a position involving more respon- 
sibility than graduates from other institutes (Additional 
file 4: Table S8). 

Not unexpectedly, respondents who graduated fairly 
recently are less likely to have had an increase in remu- 
neration; on average, they have been on the job market 
for less time than candidates who graduated before 
2008. Men appear to be more likely to have an increase 
in remuneration than women {P = 0.005), and graduates 
from HSPH, SPHFU, or UMST appear to be somewhat 
less likely to have an increase in remuneration (Additional 
file 4: Table S9). 

Recent graduates appear to be less likely to have 
switched employer than respondents who graduated 
before 2008. Furthermore, graduates from HPSH and 
SPHFU appear to be somewhat less likely to switch to 
another employer than graduates from other institutes 
(Additional file 4: Table SIO). 

Institutional differences were found with regard to 
MPH contribution to change in leadership, ^(5, 303) = 
16.217, P <0.001, ^7^ = 0.211; change in technical pos- 
ition, f (5, 303) = 19.762, P <0.001, = 0.237; increase in 
remuneration, F{5, 303) = 15.822, P <0.001, = 0.196; 
and a change in employer, f (5, 303) = 9.983, P <0.001, 



t]^ = 0.182. With regard to contribution to change in 
leadership and contribution to change in technical pos- 
ition, graduates from KIT, INSP, and SPHUWC gave 
significantly higher responses than did graduates from 
HSPH, SPHFU, and UMST. With regard to MPH con- 
tributing to change in remuneration, graduates from 
KIT and SPHUWC gave significantly higher responses 
than did graduates from the other four institutions. Fi- 
nally, with regard to attribution to change in employer, 
graduates from KIT, SPHUWC, and UMST gave higher 
responses than respondents from HSPH, SPHFU, and 
INSP. No other statistically significant predictor vari- 
ables were found. 

Institution also contributed significantly to differences 
between candidates in reported application of acquired 
competencies, F(5, 397) = 13.178, P <0.001, ?7^ = 0.166, 
with graduates from KIT, INSP, and SPHUWC responding 
significantly higher than graduates from HSPH, SPHFU, 
and UMST. Besides, graduates with a medical doctor 
background responded significantly higher than graduates 
without such a background, f(l, 397) = 8.931, i' = 0.003, 
rj^ = 0.022. The same group of institutions (KIT, INSP, and 
SPHUWC), F(5, 376) = 14.286, P <0.001, tj^ = 0.160, and 
medical doctor background, f(l, 376) = 10.278, P = 0.001, 
1]^ = 0.027, significantly contributed to differences in 
performance at the workplace, with graduates from 
SPHUWC, INSP, and KIT giving significantly higher rat- 
ings than graduates from HSPH, SPHFU, and UMST. 

Institution where graduates studied also explained part 
of the differences between candidates in increased im- 
pact on society, F{5, 403) = 11.435, P <0.001, ^/^ = 0.124, 
with graduates from KIT and SPHUWC responding 
somewhat higher than graduates from other institutions. 
Furthermore, graduates with an additional degree rated 
impact higher than graduates without an additional de- 
gree, F{1, 403) = 4.681, P = 0.031, rj'^ = 0.011. 

Finally, split- plot ANOVA suggests that, for all institu- 
tions together, the MPH program contributed slighdy more 
to application of competencies to graduates' performance 
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at the workplace than to their contribution to society, F{1, 
373) = 13.863, P <0.001, ;7^ = 0.036. However, significant 
differences between institutions were found with regard to 
this trend, f (5, 373) = 6.288, P <0.001, tj^ = 0.078. A closer 
look within institutions reveals that this trend is statisti- 
cally significant in HSPH, 108) = 31.738, P <0.001, 
tj'^ = 0.227; SPHFU, f(l, 59) = 4.230, P = 0.044, r;'^ = 
0.067; and INSP, f(l, 60) = 18.465, P <0.001, = 0.235; 
but not in SPHUWC or UMST. Finally, in KIT the trend 
appears to be reversed in that the difference is not statisti- 
cally significant, f (1, 72) = 3.561, P = 0.063, fj^ = 0.047. 

Discussion 

Our study across six MPH programs is the first to ask 
graduates for attribution to the MPH program with re- 
gard to competencies and impact variables. Our study 
reports on one of the highest numbers of graduates of 
Masters in health and health care (n = 445); the highest 
was 478; response rates reported were similar [18]. 

The study shows that, after graduation, graduates 
worked less in clinics and district health departments 
and moved to international NGOs and research insti- 
tutes. The change of work of graduates is similar to that 
reported by others [19,29,31]. Possibly, the high number 
of changes to "other" workplaces indicates a move to 
centers of disease control, which was not differentiated 
in the questionnaire. 

In contrast to the reported brain-drain of higher edu- 
cated health professionals [32-34], only 5% (24) of the 
graduates left their home country and, of these, seven left 
to work in a high income country; the remaining 17 left 
their country to work in their region of origin. Only five 
respondents, who had substantial experience in LMIC, 
came from high income countries. 

Though traditionally medical doctors enrolled for an 
MPH, a wide range of different educational backgrounds 
are represented such as a Bachelor of Public Health, 
nursing, dentistry, and social sciences [35]. 

As for career and leadership, a large proportion of 
graduates changed their leadership position, technical pos- 
ition, acquired new responsibilities, and increased their 
remuneration, and attributed these changes to the MPH 
program. In other studies change in leadership, technical 
position or new responsibilities have been reported, 
however, in those studies it was not clear whether these 
changes occurred due to gaining seniority or other factors 
[19,21,29,34,36-41]. Richardson, in 2008, was the only one 
to ask about satisfaction with professional skills and pro- 
fessional status and the contribution of the program, in 
this case occupational therapy, to that satisfaction [20]. A 
higher salary was also reported by Bradley [42], Gill [39], 
Ruth [43], Drennan [41], although attribution was unclear. 

An additional degree, as well as graduation from specific 
institutions, positively influenced change in leadership. 



technical position, and more responsibility; a medical de- 
gree positively influenced change in leadership. As for the 
institutions, having graduated from INSP, SPHUWC, or 
KIT seemed to be more beneficial to someone's career in 
terms of change in leadership, technical position, and re- 
muneration. Having graduated from INSP or KIT posi- 
tively influenced being assigned more responsibility. As 
the contexts, countries, and programs are so different, it 
is difficult to surmise what might be the reason for the 
differences between the graduates of these institutions. 
In other studies, i.e., in the USA, gender influenced in- 
crease in remuneration; this was the case in our study 
as well [42]. 

About a third of the study participants undertook fur- 
ther studies other than short courses. A high proportion 
had completed their PhD (9%), this might result from 
the time lapse between graduation and the study; others 
indicated that they were in the process of studying to- 
wards a PhD [20,34,41,44]. Other studies also reported 
graduates undertaking further studies [20,21,37,41,42]. 

In relation to the application of competencies, almost 
half of the graduates stated that the MPH program con- 
tributed substantially to the application of pubhc health 
competencies, though with large variations between insti- 
tutions. Specifically, public health analytical competencies 
as well as leadership, and context-sensitive and planning 
and management competencies were mentioned. Other 
studies reported enhanced job skills and performance [45], 
a range of public health skills, or international health com- 
petencies [29,46]. A number of studies reported specific 
skills such as management [36,37]. A number of studies 
reported generic competencies such as critical reflection 
and critical thinking, which this study did not explore 
[21,47-53]. Policy development competencies were the 
least mentioned, which may arise from, for example, the 
place of work, the degree of emphasis by specific MPH 
programs, or the different processes of policy making. 

As regards impact on the workplace, graduates attrib- 
uted the enablement by the MPH program to impact on 
the workplace between 60% and 20% for specific impact 
variables, with a large difference between institutions. 
Importantly, graduates attributed, for example, "enable- 
ment in writing a research proposal" and "reporting/ 
making recommendations on population health status", 
as this is what would be expected from a public health 
professional [54,55]. The diverse range of areas of work 
(management, research, policy, teaching) might contrib- 
ute to the fact that graduates reported, in a range of 38% 
to 14%, that each of the competencies was not used or 
not part of their work [56,57]. The highest reported vari- 
able not used or not part of their work was "contributed 
to writing a published chapter of a book", which in hind- 
sight might have been too high an expectation. Other 
studies seldom asked for impact on the workplace, or 
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asked it only qualitatively, while only two studies re- 
ported quantitatively and/or attribution [20,22]. 

Concerning impact on society, graduates reported that 
the MPH program enabled them substantially to impact 
on society with variables rated from 39% to 17% for the 
10 defined variables, with large ranges between institu- 
tions, "influencing better understanding of public health" 
and "contributing equitable access to quality services" are 
both important achievements in public health. Only two 
studies reported impact on society [20,22] and this study 
was the first which looked at defined impact. Though im- 
pact on society through higher educated professionals is 
very difficult to measure, because of many influencing fac- 
tors before, during, and after the MPH program, the re- 
sults give a good indication on what impact the graduates 
felt they did contribute. 

Limitations 

Self-reporting measures are easy to administer simultan- 
eously at different locations, they are relatively easy to 
subject to quantitative analysis, they are inexpensive, 
and are less time demanding than testing and assess- 
ment [24,26]. However, self-reported measures might be 
prone to biases such as an overly positive (i.e., loyalty of 
graduates to their MPH program) or exaggerated mod- 
esty, vagueness, and ambiguities of questions, and a ten- 
dency to give consistent evaluations across a set of 
specific items [24,26]. In this study the underlying fac- 
tors for change in leadership and new responsibilities 
were not studied. It might be that those people who 
have potential to become leaders/managers have chosen 
the MPH as a relevant training program to prepare for a 
potential higher position. On the other hand, people 
trained as MPH may show to hold competencies that 
are necessary for a leadership position, or are supposed 
to have those competencies because of the degree, so 
they tend to be promoted. Next to self-reporting as 
such, the use of different "yardsticks" across programs 
and countries (anchoring), culture, or different pro- 
grams having a different emphasis might result in bias 
[15,25,58]. However, the competencies and impact vari- 
ables had been validated across countries before the 
study [28] . In order to reduce the risk of poor anchoring 
as well as the tendency to avoid extreme responses, the 
scales were kept as small as possible. 

Using an online survey tool may have had some influ- 
ence on completeness. Graduates may have left particu- 
lar questions unanswered because of loss of connectivity 
with the institution or the online tool before or during 
the completion of the survey. Though the relatively low 
response rate influences the results of the study, as those 
who answered may have been more positive, the response 
rate of this study did not differ much from the response 
rate of other alumni surveys [28]. Efforts were exerted to 



find as many graduates as possible and to encourage them 
to fill in the survey. 

The questionnaire could have been constructed differ- 
ently by mixing items to reduce the halo error, however, 
that could have negatively impacted on the user- 
friendliness of the questionnaire. In order to increase 
validity, the questionnaire was pretested with graduates 
from all MPH programs and adjusted. The fact that the 
MPH program contributed less to the impact factors on 
society than the impact factors on the workplace can be 
seen as a measure of predictive validity as one would 
expect that graduates have less influence on society than 
on their workplace. As previously mentioned, other 
methods, such as semi-or unstructured interviews, could 
also have been used. This study is part of a larger research 
project, in which next to the alumni survey, 10 graduates 
per school, their peers, and their supervisors were inter- 
viewed. These results are being analyzed. 

Conclusions 

This is the first transnational study on outcome and im- 
pact of MPH programs and the first transnational study 
on a Master's in health and health care. From this study it 
can be concluded that, according to graduates, the MPH 
programs contribute to improvements in graduates' ca- 
reers and to leadership building in public health. The 
MPH programs contributed substantially to the applica- 
tion of public health analytical competencies as well as 
leadership, context-specific and planning and manage- 
ment competencies. Graduates reported substantial con- 
tribution by the MPH program on impact variables on the 
workplace such as development of a research proposal 
and reporting on population health needs. The contribu- 
tion to impact variables on society, such as "contributing 
to equitable access to quality services", was less. The dif- 
ferences between the MPH programs from different coun- 
tries warrant further study in order to find explanations. It 
is argued that this study makes some progress in problem- 
atizing and measuring impact of MPH programs for the 
first time. It is also concluded that the follow-up of gradu- 
ates, as performed in this study, is an efficient and prac- 
tical way to reach a large number of respondents across 
countries and could be readily replicated. The results of 
the in-depth study are still to follow. Further strategies to 
enhance understanding of impact might be focus group 
discussions with graduates, though more costly and logis- 
tically difficult, or an employer survey. Finally, the findings 
could, and will be in the cases of the participating insti- 
tutions, used to steer curriculum reform and innovation. 
For example "policy development competencies" were 
assessed as lowly attributed to the MPH program, and 
therefore curricula could include knowledge and skills 
building around analyzing, evaluating, and developing 
policy options for public health programs. 
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